Preprocessing and Feature Extraction Techniques for Multimodal Interactive Transcription of Text Images

To date, automatic handwriting recognition systems are far from being perfect and heavy human intervention is often required to check and correct the results of such systems. This “post-editing” process is both inefficient and uncomfortable to the user. An example is the transcription of historic documents: State-of-the-art handwritten text recognition technology is not suitable to perform this task automatically and expensive paleography expert work is needed to achive correct transcriptions. As an alternative to post-editing, a multimodal interactive approach is proposed here, where user feedback is provided by means of touch-screen pen strokes and/or more traditional keyboard and mouse operation. User’s feedback directly allows to improve system accuracy, while multimodality increases system ergonomy and user acceptability. Multimodal interaction is approached in such a way that both the main and the feedback data streams help each-other to optimize overall performance and usability. Empirical tests on three cursive handwritten tasks suggest that, using this approach, significant amounts of user effort can be saved with respect to the conventional, non-interactive, post-editing process.